41 research outputs found
Deep learning & remote sensing : pushing the frontiers in image segmentation
Dissertação (Mestrado em Informática) — Universidade de BrasÃlia, Instituto de Ciências Exatas, Departamento de Ciência da Computação, BrasÃlia, 2022.A segmentação de imagens visa simplificar o entendimento de imagens digitais e métodos de
aprendizado profundo usando redes neurais convolucionais permitem a exploração de diferentes
tarefas (e.g., segmentação semântica, instância e panóptica). A segmentação semântica atribui
uma classe a cada pixel em uma imagem, a segmentação de instância classifica objetos a nÃvel
de pixel com um identificador exclusivo para cada alvo e a segmentação panóptica combina
instâncias com diferentes planos de fundo. Os dados de sensoriamento remoto são muito adequados para desenvolver novos algoritmos. No entanto, algumas particularidades impedem que o
sensoriamento remoto com imagens orbitais e aéreas cresça quando comparado à s imagens tradicionais (e.g., fotos de celulares): (1) as imagens são muito extensas, (2) apresenta caracterÃsticas
diferentes (e.g., número de canais e formato de imagem), (3) um grande número de etapas de préprocessamento e pós-processamento (e.g., extração de quadros e classificação de cenas grandes) e
(4) os softwares para rotulagem e treinamento de modelos não são compatÃveis. Esta dissertação
visa avançar nas três principais categorias de segmentação de imagens. Dentro do domÃnio de
segmentação de instâncias, propusemos três experimentos. Primeiro, aprimoramos a abordagem
de segmentação de instância baseada em caixa para classificar cenas grandes. Em segundo
lugar, criamos um método sem caixas delimitadoras para alcançar resultados de segmentação
de instâncias usando modelos de segmentação semântica em um cenário com objetos esparsos.
Terceiro, aprimoramos o método anterior para cenas aglomeradas e desenvolvemos o primeiro
estudo considerando aprendizado semissupervisionado usando sensoriamento remoto e dados
GIS. Em seguida, no domÃnio da segmentação panóptica, apresentamos o primeiro conjunto de
dados de segmentação panóptica de sensoriamento remoto e dispomos de uma metodologia para
conversão de dados GIS no formato COCO. Como nosso primeiro estudo considerou imagens
RGB, estendemos essa abordagem para dados multiespectrais. Por fim, melhoramos o método
box-free inicialmente projetado para segmentação de instâncias para a tarefa de segmentação
panóptica. Esta dissertação analisou vários métodos de segmentação e tipos de imagens, e as
soluções desenvolvidas permitem a exploração de novas tarefas , a simplificação da rotulagem
de dados e uma forma simplificada de obter previsões de instância e panópticas usando modelos
simples de segmentação semântica.Image segmentation aims to simplify the understanding of digital images. Deep learning-based
methods using convolutional neural networks have been game-changing, allowing the exploration
of different tasks (e.g., semantic, instance, and panoptic segmentation). Semantic segmentation
assigns a class to every pixel in an image, instance segmentation classifies objects at a pixel
level with a unique identifier for each target, and panoptic segmentation combines instancelevel predictions with different backgrounds. Remote sensing data largely benefits from those
methods, being very suitable for developing new DL algorithms and creating solutions using
top-view images. However, some peculiarities prevent remote sensing using orbital and aerial
imagery from growing when compared to traditional ground-level images (e.g., camera photos):
(1) The images are extensive, (2) it presents different characteristics (e.g., number of channels
and image format), (3) a high number of pre-processes and post-processes steps (e.g., extracting
patches and classifying large scenes), and (4) most open software for labeling and deep learning applications are not friendly to remote sensing due to the aforementioned reasons. This
dissertation aimed to improve all three main categories of image segmentation. Within the instance segmentation domain, we proposed three experiments. First, we enhanced the box-based
instance segmentation approach for classifying large scenes, allowing practical pipelines to be
implemented. Second, we created a bounding-box free method to reach instance segmentation
results by using semantic segmentation models in a scenario with sparse objects. Third, we
improved the previous method for crowded scenes and developed the first study considering
semi-supervised learning using remote sensing and GIS data. Subsequently, in the panoptic
segmentation domain, we presented the first remote sensing panoptic segmentation dataset containing fourteen classes and disposed of software and methodology for converting GIS data into
the panoptic segmentation format. Since our first study considered RGB images, we extended
our approach to multispectral data. Finally, we leveraged the box-free method initially designed
for instance segmentation to the panoptic segmentation task. This dissertation analyzed various
segmentation methods and image types, and the developed solutions enable the exploration of
new tasks (such as panoptic segmentation), the simplification of labeling data (using the proposed semi-supervised learning procedure), and a simplified way to obtain instance and panoptic
predictions using simple semantic segmentation models
Automatic detection of roads and pivot tracks using high spatial resolution images from PRISM-ALOS sensor
Este artigo apresenta um novo método para a detecção de estradas e trilhas de pivôs em imagens pancromáticas de alta resolução. A abordagem é dividida em três módulos principais: primeiro, uma filtragem espacial é aplicada a imagem para a extração de magnitudes e direções de descontinuidades, então uma quantização em 5 nÃveis é aplicada à s magnitudes e um algoritmo de consistência hierárquica é desenvolvido para checar e unir cada janela a sua vizinhança, priorizando direções com os maiores e mais consistentes valores de magnitude; segundo, as hipóteses mais fortes são mantidas enquanto as mais fracas são retiradas da imagem, com o intuito de eliminar partes desconectadas; e finalmente, a transformada de Hough é aplicada nas hipóteses para completar as estradas detectadas e também identificar possÃveis trilhas de pivôs entre elas. Este método foi aplicado em imagens do satélite ALOS em d iferentes áreas do oeste do estado da Bahia, Brasil, e os resultados são fornecidos aqui. _________________________________________________________________________________ ABSTRACTThis article introduces a new method for road and pivot tracks detection for high resolution pancromatic satellite
images. The approach is divided on three main modules: firstly, a spatial derivative mask is applied to the image for
extraction of magnitudes and directions of discontinuities, then a five level quantization is applied on the magnitudes
and a hierarchical consistency algorithm is developed by checking and linking each window to its neighborhood
prioritizing directions with higher and consistent magnitude values; secondly, the road stronger hypotheses are left
while the weaker ones are cleaned from the images, with the purpose to eliminate disconnected pieces; and finally, a
Hough transformation is applied on the road hypotheses in order to complete the roads detected and to also identify
possible pivot tracks among them. The method was applied to images of the ALOS satellite of different areas of the
west of state of Bahia, Brazil, and results are provided here
Bounding Box-Free Instance Segmentation Using Semi-Supervised Learning for Generating a City-Scale Vehicle Dataset
Vehicle classification is a hot computer vision topic, with studies ranging
from ground-view up to top-view imagery. In remote sensing, the usage of
top-view images allows for understanding city patterns, vehicle concentration,
traffic management, and others. However, there are some difficulties when
aiming for pixel-wise classification: (a) most vehicle classification studies
use object detection methods, and most publicly available datasets are designed
for this task, (b) creating instance segmentation datasets is laborious, and
(c) traditional instance segmentation methods underperform on this task since
the objects are small. Thus, the present research objectives are: (1) propose a
novel semi-supervised iterative learning approach using GIS software, (2)
propose a box-free instance segmentation approach, and (3) provide a city-scale
vehicle dataset. The iterative learning procedure considered: (1) label a small
number of vehicles, (2) train on those samples, (3) use the model to classify
the entire image, (4) convert the image prediction into a polygon shapefile,
(5) correct some areas with errors and include them in the training data, and
(6) repeat until results are satisfactory. To separate instances, we considered
vehicle interior and vehicle borders, and the DL model was the U-net with the
Efficient-net-B7 backbone. When removing the borders, the vehicle interior
becomes isolated, allowing for unique object identification. To recover the
deleted 1-pixel borders, we proposed a simple method to expand each prediction.
The results show better pixel-wise metrics when compared to the Mask-RCNN (82%
against 67% in IoU). On per-object analysis, the overall accuracy, precision,
and recall were greater than 90%. This pipeline applies to any remote sensing
target, being very efficient for segmentation and generating datasets.Comment: 38 pages, 10 figures, submitted to journa
Comparação dos métodos green e atrem para correção atmosférica de imagens hiperespectrais aviris
O presente trabalho tem como propósito realizar uma análise comparativados métodos Green e ATREM para a correção atmosférica de imagens do sensor hiperespectral AVIRIS. Também foi avaliada a aplicação do método complementar EFFORT, que proporciona uma filtragem de eventuais resÃduos atmosféricos verificadosapós a correção. Verificou-se que o método Green apresentou melhores resultados emcomparação ao método ATREM. O emprego do EFFORT permitiu uma melhora dosespectros do ATREM, porém apresentou apenas uma melhora moderada sobre os resultados do método Green. _______________________________________________________________________________ ABSTRACTThe present work deals with a comparative analysis of two atmospheric correction methods for AVIRIS data: the Green and the ATREM methods. It is also analysed the procedure EFFORT, associated to the atmospheric correction, which is applied after the correction in order to eventually filter atmospheric residues. It is shown that the Green’s Method presented better results, compared to the ATREM Method. The use of the EFFORT procedure produced a significant improvement on the ATREM’s results, but the improvement on the Green’s Method results was moderate
Panoptic Segmentation Meets Remote Sensing
Panoptic segmentation combines instance and semantic predictions, allowing
the detection of "things" and "stuff" simultaneously. Effectively approaching
panoptic segmentation in remotely sensed data can be auspicious in many
challenging problems since it allows continuous mapping and specific target
counting. Several difficulties have prevented the growth of this task in remote
sensing: (a) most algorithms are designed for traditional images, (b) image
labelling must encompass "things" and "stuff" classes, and (c) the annotation
format is complex. Thus, aiming to solve and increase the operability of
panoptic segmentation in remote sensing, this study has five objectives: (1)
create a novel data preparation pipeline for panoptic segmentation, (2) propose
an annotation conversion software to generate panoptic annotations; (3) propose
a novel dataset on urban areas, (4) modify the Detectron2 for the task, and (5)
evaluate difficulties of this task in the urban setting. We used an aerial
image with a 0,24-meter spatial resolution considering 14 classes. Our pipeline
considers three image inputs, and the proposed software uses point shapefiles
for creating samples in the COCO format. Our study generated 3,400 samples with
512x512 pixel dimensions. We used the Panoptic-FPN with two backbones
(ResNet-50 and ResNet-101), and the model evaluation considered semantic
instance and panoptic metrics. We obtained 93.9, 47.7, and 64.9 for the mean
IoU, box AP, and PQ. Our study presents the first effective pipeline for
panoptic segmentation and an extensive database for other researchers to use
and deal with other data or related problems requiring a thorough scene
understanding.Comment: 40 pages, 10 figures, submitted to journa
COMPARAÇÃO DOS MÉTODOS GREEN E ATREM PARA CORREÇÃO ATMOSFÉRICA DE IMAGENS HIPERESPECTRAIS AVIRIS
The present work deals with a comparative analysis of two atmospheric correction methods for AVIRIS data: the Green and the ATREM methods. It is also analysed the procedure EFFORT, associated to the atmospheric correction, which is applied after the correction in order to eventually filter atmospheric residues. It is shown that the Green’s Method presented better results, compared to the ATREM Method. The use of the EFFORT procedure produced a significant improvement on the ATREM’s results, but the improvement on the Green’s Method results was moderate.O presente trabalho tem como propósito realizar uma análise comparativa dos métodos Green e ATREM para a correção atmosférica de imagens do sensor hiperespectral AVIRIS. Também foi avaliada a aplicação do método complementar EFFORT, que proporciona uma filtragem de eventuais resÃduos atmosféricos verificados após a correção. Verificou-se que o método Green apresentou melhores resultados em comparação ao método ATREM. O emprego do EFFORT permitiu uma melhora dos espectros do ATREM, porém apresentou apenas uma melhora moderada sobre os resultados do método Green
Rice crop detection using LSTM, Bi-LSTM, and machine learning models from sentinel-1 time series
The Synthetic Aperture Radar (SAR) time series allows describing the rice phenological cycle by the backscattering time signature. Therefore, the advent of the Copernicus Sentinel-1 program expands studies of radar data (C-band) for rice monitoring at regional scales, due to the high temporal resolution and free data distribution. Recurrent Neural Network (RNN) model has reached state-of-the-art in the pattern recognition of time-sequenced data, obtaining a significant advantage at crop classification on the remote sensing images. One of the most used approaches in the RNN model is the Long Short-Term Memory (LSTM) model and its improvements, such as Bidirectional LSTM (Bi-LSTM). Bi-LSTM models are more effective as their output depends on the previous and the next segment, in contrast to the unidirectional LSTM models. The present research aims to map rice crops from Sentinel-1 time series (band C) using LSTM and Bi-LSTM models in West Rio Grande do Sul (Brazil). We compared the results with traditional Machine Learning techniques: Support Vector Machines (SVM), Random Forest (RF), k-Nearest Neighbors (k-NN), and Normal Bayes (NB). The developed methodology can be subdivided into the following steps: (a) acquisition of the Sentinel time series over two years; (b) data pre-processing and minimizing noise from 3D spatial-temporal filters and smoothing with Savitzky-Golay filter; (c) time series classification procedures; (d) accuracy analysis and comparison among the methods. The results show high overall accuracy and Kappa (>97% for all methods and metrics). Bi-LSTM was the best model, presenting statistical differences in the McNemar test with a significance of 0.05. However, LSTM and Traditional Machine Learning models also achieved high accuracy values. The study establishes an adequate methodology for mapping the rice crops in West Rio Grande do Sul
Remote sensing for monitoring photovoltaic solar plants in Brazil using deep semantic segmentation
Brazil is a tropical country with continental dimensions and abundant solar resources that are still underutilized. However, solar energy is one of the most promising renewable sources in the country. The proper inspection of Photovoltaic (PV) solar plants is an issue of great interest for the Brazilian territory’s energy management agency, and advances in computer vision and deep learning allow automatic, periodic, and low-cost monitoring. The present research aims to identify PV solar plants in Brazil using semantic segmentation and a mosaicking approach for large image classification. We compared four architectures (U-net, DeepLabv3+, Pyramid Scene Parsing Network, and Feature Pyramid Network) with four backbones (Efficient-net-b0, Efficient-net-b7, ResNet-50, and ResNet-101). For mosaicking, we evaluated a sliding window with overlapping pixels using different stride values (8, 16, 32, 64, 128, and 256). We found that: (1) the models presented similar results, showing that the most relevant approach is to acquire high-quality labels rather than models in many scenarios; (2) U-net presented slightly better metrics, and the best configuration was U-net with the Efficient-net-b7 encoder (98% overall accuracy, 91% IoU, and 95% F-score); (3) mosaicking progressively increases results (precision-recall and receiver operating characteristic area under the curve) when decreasing the stride value, at the cost of a higher computational cost. The high trends of solar energy growth in Brazil require rapid mapping, and the proposed study provides a promising approach
A Data-Centric Approach for Wind Plant Instance-Level Segmentation Using Semantic Segmentation and GIS
Wind energy is one of Brazil’s most promising energy sources, and the rapid growth of wind plants has increased the need for accurate and efficient inspection methods. The current onsite visits, which are laborious and costly, have become unsustainable due to the sheer scale of wind plants across the country. This study proposes a novel data-centric approach integrating semantic segmentation and GIS to obtain instance-level predictions of wind plants by using free orbital satellite images. Additionally, we introduce a new annotation pattern, which includes wind turbines and their shadows, leading to a larger object size. The elaboration of data collection used the panchromatic band of the China–Brazil Earth Resources Satellite (CBERS) 4A, with a 2-m spatial resolution, comprising 21 CBERS 4A scenes and more than 5000 wind plants annotated manually. This database has 5021 patches, each with 128 × 128 spatial dimensions. The deep learning model comparison involved evaluating six architectures and three backbones, totaling 15 models. The sliding windows approach allowed us to classify large areas, considering different pass values to obtain a balance between performance and computational time. The main results from this study include: (1) the LinkNet architecture with the Efficient-Net-B7 backbone was the best model, achieving an intersection over union score of 71%; (2) the use of smaller stride values improves the recognition process of large areas but increases computational power, and (3) the conversion of raster to polygon in GIS platforms leads to highly accurate instance-level predictions. This entire pipeline can be easily applied for mapping wind plants in Brazil and be expanded to other regions worldwide. With this approach, we aim to provide a cost-effective and efficient solution for inspecting and monitoring wind plants, contributing to the sustainability of the wind energy sector in Brazil and beyond
A Data-Centric Approach for Rapid Dataset Generation Using Iterative Learning and Sparse Annotations
This study investigates the application of iterative sparse annotations for semantic segmentation in remote-sensing imagery, focusing on minimizing the laborious and expensive data labeling process. By leveraging Geographic Information Systems (GIS), we implemented circular polygon shapefiles to label portions of each class, attributing a value of -1 outside these polygons. The model training used the simplified BSB Aerial Dataset with eight classes. The semantic segmentation model was U-Net architecture with the Efficient-net-B7 backbone and a modified cross-entropy loss function. Our results showed promising improvement, particularly in error-prone classes, with the iterative addition of more samples. This approach suggests a quicker method for dataset creation using sparse, iteratively enhanced annotations. Future work will aim to implement further iterative rounds to approximate the results of continuous labeling, thereby enhancing the efficiency of semantic segmentation in large-scale remote-sensing images